BRUHAT INTERVALS AS ROOKS ON SKEW FERRERS BOARDS 
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Abstract. We characterise the permutations 7r such that the elements in the 
closed lower Bruhat interval [id, tt] of the symmetric group correspond to non- 
taking rook configurations on a skew Ferrers board. It turns out that these are 
exactly the permutations tt such that [id, n] corresponds to a flag manifold defined 
by inclusions, studied by Gasharov and Reiner. 

Our characterisation connects the Poincare polynomials (rank-generating func- 
tion) of Bruhat intervals with g-rook polynomials, and we are able to compute the 
Poincare polynomial of some particularly interesting intervals in the finite Weyl 
groups A n and B n . The expressions involve q-Stirling numbers of the second 
kind. 

As a by-product of our method, we present a new Stirling number identity 
connected to both Bruhat intervals and the poly-Bernoulli numbers defined by 
Kaneko. 



1. Introduction 

Since its introduction in the 1930s the Bruhat order on Coxeter groups has at- 
tracted mathematicians from many areas. Geometrically it describes the contain- 
ment ordering of Schubert varieties in flag manifolds and other homogeneous spaces. 
Algebraically it is intimately related to the representation theory of Lie groups. 
Combinatorially the Bruhat order is essentially the subword order on reduced words 
in the alphabet of generators of a Coxeter group. 

The interval structure of the Bruhat order is geometrically very important and 
has been studied a lot in the literature. From a combinatorial point of view, as soon 
as there is a (graded) poset, the following three questions naturally arise about its 
intervals [u,w] (and they will probably arise in the following order): 

(1) What is the rank-generating function (or Poincare polynomial) Poinj U U) ] (q) = 

V r , 

(2) What is the Mobius function fi(u,w)? 

(3) What can be said about the topology of the order complex of (u,w)7 

The third question was answered by Bjorner and Wachs [3_ in 1982: The order 
complex of an open interval (u,w) is homeomorphic to the sphere §;^( M > U ')~ 2 . The 
second question was answered already by Verma [IS] in 1971: fj,(u,w) = (— \)^ u ' w \ 
However, the first question is still a very open problem! 

For the whole poset Poin((7) was computed by Steinberg ^7j, Chevalley and 
Solomon |16j . Really small intervals (of length < 7 in A n and ^ 5 in B n and 
D n ) were completely classified by Hultman and Incitti [12]. Lower intervals of 
312-avoiding permutations in A n were classified by Develin jB] (though he did not 
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compute their Poincare polynomials), and for a general lower interval [id, w] in a 
crystallographic Coxeter group, Bjorner and Ekedahl [2j showed that the coefficients 
of Poinry^i (q) are partly increasing. Apart from this, virtually nothing is known, 
not even for lower intervals. 

The aim of this paper is to start filling the hole and at least gain some understand- 
ing of the rank-generating function of a family of intervals in finite Weyl groups. 
To this end we present a connection between the Poincare polynomial Poin^) and 
rook polynomials, making it possible to compute Poin((7) for various interesting in- 
tervals. Our approach is partly a generalisation of the notion of partition varieties 
introduced by Ding [7] to what may be called skew partition varieties. 

The paper is composed as follows. In Section [21 we give a short introduction to 
rook polynomials before presenting our results in Section |21 In Section 0] we present 
the connection between rook polynomials and Poincare polynomials and prove our 
main theorem. In sections [S] and we apply our main theorem to intervals in the 
symmetric group A n . As a by-product a new Stirling number identity pops up at the 
end of Section In Section [7| we apply our main theorem to the hyperoctahedral 
group B n . Finally, in Section |H] we discuss further research directions and suggest 
some open problems. 

2. Rook polynomials 

Let A be a zero-one matrix and put rooks on some of the one-entries of A. If no 
two rooks are in the same row or column we have a (non-taking) rook configuration 
on A, and we say that A covers the rook configuration. In the literature, A is 
sometimes called a board and is often depicted by square diagrams like those in 
Figure n F° r convenience we will simultaneously think of A as the set of its one- 
entries, and write for instance (i,j) € A if A4 j = 1 and use notation like An B. 

Let A$ and A^ denote reflecting the matrix upside down respectively rotating it 
180 degrees, i.e. A\ ■ = A m _j + ij and = Atn-i+i^n-j+i if A is an m x n matrix. 
Define tv* and tt^ similarly for rook configurations ir. 

The number of rook configurations on A with k rooks is called the kth rook number 
of A and is denoted by R^. Given a nonnegative integer n, following Goldman et 
al. jlUj we define the nth rook polynomial of A as 

n 

Rnix) = £ B£- k x{x - 1) ■ • • (X - k + 1). 

A:=0 

Note that r£(0) = R%. 

A zero-one matrix A is a left-aligned (resp. right- aligned) Ferrers matrix if every 
one-entry has one-entries directly to the left (resp. to the right) and above it (pro- 
vided these entries exist). The number of ones in the ith row (resp. column) of A is 
denoted by n(A) (resp. q(A)). Figure^shows an example. 

From JOJ we have the following theorem. 

Theorem 2.1 (Goldman et al.). Let X be a right-aligned Ferrers matrix of size 
m x n. Then 

n 

Rii(.x) = Y[(x+c J (\)-j + i). 

i=i 
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Figure 1. Square diagrams of the left-aligned Ferrers matrix A 
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and the right-aligned Ferrers matrix \x 



and column lengths are given by the following table: 
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Given a rook configuration .A on A, define the statistics iiwa(A) to be the number 
of (not necessarily positive) cells of A with no rook weakly to the right in the same 
row or below in the same column. In the special case where A is an n x n matrix 
and A has n rooks, inv a(A) becomes the number of inversions of the permutation 
7r given by ir(i) = j 4^ £ A, where i is the row index and j is the column 

index. 

Next, (almost) following Garsia and Remmel [Sj, we define the kth q-rook number 
of A as 

A 

where the sum is over all rook configurations on A with k rooks. Given a nonnegative 
integer n, the nth q-rook polynomial of A is defined as 

n 

Rn(x;q) = XX-fcGrtbU* -i] q ---[x-k + i) q . 

k=0 

Here [x] q A = 1 + q + q 2 + • • • + q x ^ = (1 — q x ) /(l — q) is the g-analogue of x. Observe 
that putting q = 1 yields the ordinary rook numbers and polynomials. 

Garsia and Remmel showed that Theorem 12 . 1 1 has a beautiful (/-analogue: 

Theorem 2.2 (Garsia, Remmel). Let A be a left-aligned Ferrers matrix of size 
m x n. Then 

n 

R%{x;q) =q z Y[[x + Cj (A)+j -n] q 

where z is the number of zero-entries in A. 

Let [n]\ q ^[l] q [2] q ---[n] q . 

Corollary 2.3. For the n x n square matrix J n ' n with ones everywhere, the nth 
q-rook number is 

R J n n ' n (q) = [n]V 

Let T n denote the n x n zero-one matrix with ones on and above the secondary 
diagonal, i.e. (T n )ij = 1 4$ % < n — j + 1. In p. 248] it is proved that 

(1) R T k n {q) =q®S n+1>n+1 - k (q) 



-1 



JONAS SJOSTRAND 



Figure 2. The shaded regions show the left respectively right hull 
of the permutation 35124. 



where S nj k(q) is the q-Stirling number defined by the recurrence 

S n+ x,k{q) = q k ~ l S nt k-i + [k] q S ni k(q), for < k < n 
with the initial conditions So t o(q) = 1 and S n k = for k < or k > n. 

3. Results 

A skew Ferrers matrix X/fJ, is the difference X — fj, between a Ferrers matrix A 
and an equally aligned componentwise smaller Ferrers matrix fi. If A and \i are 
left-aligned, then Xj [i is also said to be left-aligned, and if A and \i are right- aligned, 
so is X/ is. 

Let & n denote the symmetric group. For any zero-one n x n matrix A, let &(A) 
be the set of rook configurations on A with n rooks. We will identify such a rook 
configuration with a permutation tt € 6„ so that ir(i) = j if and only if there is a 
rook at the square where % is the row index and j is the column index. 

For a permutation tt £ 6 n , let the right (resp. left) hull Hr(tt) (resp. Hl{tt)) of 
7r be the smallest right-aligned (resp. left-aligned) skew Ferrers matrix that covers 
7r. Figure 12 shows an example. 

For the definition of Bruhat order and a general treatment of Coxeter groups from 
a combinatorialist's viewpoint, we refer to Bjorner and Brenti pQ. 

Our main result is the following theorem and its corollary. 

Theorem 3.1. &(Hft{n)) equals the lower Bruhat interval [id, tt] in & n if and only 
if tv avoids the patterns 4231, 35142, 42513, and 351624. 

Remark 3.2. The permutations it in the theorem are exactly the ones such that 
the Schubert variety corresponding to the interval [id, tt] is defined by inclusions in 
the sense of Gasharov and Reiner |Hj according to their Theorem 4.2. 

Corollary 3.3. Let u, w E & n and suppose w and iJ both avoid the patterns 4231, 
35142, 42513, and 351624. Then the following holds. 

(1) <5(Hr(w) fl Hl(u)) equals the Bruhat interval [u,w]. 

(2) The Poincare polynomial Pom^^g) of [u,w] equals the q-rook number 

(3) In particular, the number of elements in [u, w] equals the ordinary rook num- 
berR^ w)nHLiu) . 

Proof. Once we observe that Hl(u) = Hr(u^)^ and recall that flipping the rook 
configurations upside down is an antiautomorphism on the Bruhat order on <3 n , the 
corollary follows directly from Theorem 13 .11 □ 

Remark 3.4. If tt is 312-avoiding then Hr(tt) is an ordinary Ferrers matrix and 
Corollary 13.31 overlaps with Theorem 33 in [7j by Ding. In this case Ding coined the 
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Figure 3. Left: The permutations u = 562314978 (dots) and 
w = 687594123 (circles) in 69 satisfy the pattern condition in 
Corollary 13.31 so the interval [u, w] consists of precisely the permu- 
tations that fit inside the shaded region Hr(w) n Hl(u). Right: 
# R (56781234) n H L (43218765) is the Aztec diamond of order 4. (In 
fact it follows from part (3) of Corollary 13.31 that there are 2 n ele- 
ments in the interval [u^io] in A2 n -i, where w = max Ajn— " .) 



B n 



s\ s 2 S3 

. . . 

So Si s 2 



Sn—l S n 



Sn-2 S n -i 



Figure 4. The Coxeter graphs of A n and B, : 



name partition variety for the Schubert variety corresponding to the Bruhat interval 
[id, 7f] in & n . Thus it would be logical to coin the name skew partition variety for a 
Schubert variety corresponding to an interval [id, it] such that [id, tt] = &(Hr(tt)). 

Figure shows two examples of the corollary. 

Nontrivial application of the above result yields the Poincare polynomial of some 
particularly interesting intervals in finite Weyl groups. 

For a Coxeter system (W, S) and a subset J C S of the generators, let Wj denote 
the parabolic subgroup generated by J. Each left coset wWj G W/Wj has a unique 
representative of minimal length, see ^ Cor. 2.4.5]. The system of such minimal 
coset representatives is denoted by W J , and the Bruhat order on W restricts to an 
order on W J . 

We will deal with two infinite families of finite Coxeter systems, namely the 
symmetric groups A n and the hyperoctahedral groups B n . Their Coxeter graphs 
are depicted in Figure EJ 

For type A we have the following result. 

Theorem 3.5. Let w be the maximal element of A^^^ k \ Then the Poincare poly- 
nomial of the Bruhat interval [id, w] is 

k 

Poin [ldiH (g) = q in ~ k)k 5fc+i,i+i(l/?) S„_ fe+1)i+1 (l/<z) q\ 

i=0 
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The special case where n is even, k = n/2, and q = 1 follows from Exercise 4.36 
in Lovasz |14j once one knows that the set of permutations he describes is a Bruhat 
interval (which is Exercise 2.6 in pQ). See also Theorem 3 in Vesztergombi |19j . 

For type B the corresponding result looks like this: 

Theorem 3.6. Let w be the maximal element of Bn^ s °^ ■ Then the Poincare poly- 
nomial of the Bruhat interval [id, w] is 

i n 

Poin[ id) ™](g) = qV$ )^2S n+lti+1 (l/q) [i]\ q . 

i=0 

We also present a recurrence relation for computing the number of elements in the 
Bruhat interval [id, w] of A n ^\ where w is any element in A^)j^ k \ As a by-product 
we obtain the following Stirling number identity which appears to be new. 

Theorem 3.7. Let w be the maximal element of A^^ k \ Then the number of 
elements in the Bruhat interval [id, w] is 

k 



P° m [id,u;](l) — S n -k+l,i+l i 



|2 

i=0 



A 



(- 1 ) fc E(- 1 ) i ( i + 1 ) n " fci! ^ 



i=0 

k—n 

k 



where S n k are the Stirling numbers of the second kind, and B k are the poly-Bernoulli 
numbers defined by Kaneko |13j . 

Remark 3.8. Kaneko's Theorem 2 says that B~ k = B^ n for any n, k > 0. This is 
immediately evident from our new formula for the poly-Bernoulli numbers. 

From Kaneko's work |13l p. 223] we can compute the exponential bivariate gen- 
erating function for Poin^^l). 

x n y n ~ k e x+y 
^^ F ° m[id > w] ^ri (n - k)l = e * + ev-e*+y' 

n=0A:=0 v ; 

The poly-Bernoulli numbers have the sequence number A099594 in Sloane's On-Line 
Encyclopedia of Integer Sequences. 

4. Skew Ferrers matrices and Poincare polynomials 

In this section we make a connection between Poincare polynomials and rook 
polynomials, and prove Theorem 13.11 

Proposition 4.1. If A//i is a right-aligned skew Ferrers matrix of size n x n, then 
&(X/fi) is an order ideal in the Bruhat order of & n . 

Proof. The Bruhat order is the transitive closure of the Bruhat graph whose edges 
correspond to transpositions (see ^ Sec. 2.1]). Thus it suffices to show that we 
cannot leave A//x by a transposition going down in the Bruhat order. In other 
words, if 7r is a rook configuration on A//x with n rooks, and 7Tj > 7Tj/ with i < i', 
then exchanging rows i and i' yields a rook configuration which is covered by A//i. 
This is obviously true, as we can see in Figure |SJ □ 
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Figure 5. The permutation 35124 to the left becomes 32154 to the 
right after exchanging rows 2 and 4. We do not leave the shaded 
region A/// by this operation. 



For 7r £ & n and i, j G [n] ={1,2,..., n}, let 

d = {a £ [i] : vr(a) > j}. 

In other words 7r[i,j] is the number of rooks weakly north-east of the square (i, j). 
The following criterion for comparing two permutations with respect to the Bruhat 
order is well-known (see e.g. Th. 2.1.5]). 

Lemma 4.2. Let ir,p € & n . Then it < p if and only if ir[i,j] < p[i,j] for all 
i,j e [n]. 

Theorem 13.11 completely characterises the interesting cases where &(X/p) is a 
lower Bruhat interval [id, 7r] . Now we are ready for the proof. 

Proof of Theorem 13. 1L We begin with the "only if" direction which is the easier 
one. For each of the four forbidden patterns we will do the following: First we 
suppose 7r contains the pattern. Then we move some of the rooks that constitute 
the pattern to new positions, and call the resulting rook configuration p. This p is 
seen to be covered by H^ir) while p ir in Bruhat order, and we conclude that tt 
is not uniquely maximal in Hr(jt). 

Suppose 7r contains the pattern 4231 so that there are rooks (i2,j2), 
(hija), and (u,ji) with i\ < %2 < h < ii and j\ < ji < h < H- Move the rooks 
fe ; j2) and (^3,^3) to the positions (^2,^3) and (^3, j'2) and call the resulting rook 
configuration p. Then p is covered by Hr(it) and p > n in Bruhat order so 7r is not 
maximal in Hr(it). 

Note that the rooks outside the pattern turned out to be irrelevant for the dis- 
cussion. In fact we could have supposed ir was equal to the pattern 4231 and then 
simply defined p = 4321. This observation applies to the remaining three patterns 
as well, and thus the "only if" part of the proof can be written as a table that 
associates a p to each pattern ir: 



IT 


P 


4231 


4321 


35142 


15432 


42513 


43215 


351624 


154326 



Figure illustrates the table and makes it evident that p is covered by Hji(tt) in 
each case. That p j£ tt in Bruhat order can be checked easily using Lemma 14.21 

Now it is time to prove the difficult "if" direction. Suppose S(-fT^(7r)) ^ [id, n] 
so that there is a p € &(Hjt(ir)) with p ^ vr. Our goal is to show that tt contains 
some of the four forbidden patterns. 
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Figure 6. The dots show the rook configuration tt in the four cases 
4231, 35142, 42513, and 351624. The shaded squares show the right 
hull Hr(it), and the circles show p. 



J max 

I 



Figure 7. The shaded region contains no black rook. 



Let the rooks of tt and p be black and white, respectively. (Observe that some 
squares may contain both a black and a white rook.) Order the squares [n] 2 partially 
so that < if £ < £' and j > j' , i.e. the north-east corner (l,n) is the 

minimal square of [n] 2 . 

Let L be the set of squares with p[i,j] > ir[i,j] and no black rook weakly to 
the right of in row i or above (£, j) in column j. First we show that L is not 
empty. 

Since p ^ tt, by Lemma l4~2l there is a square G [n] 2 such that j] > n[i, j]. 
Let (£ mm , jmin) be a minimal square with this property. Then there is no black rook 
weakly to the right of (i m in, jmin) in row i mm , for if that were the case the smaller 
square (£ min - 1, j min ) would have the property p[i min - l,j m i n ] > vr[z min - 1, j min ] as 
well. Analogously, there is no black rook weakly above (z m i n , j m in) in column j m i n . 
Thus (imin, jmin) belongs to L. 

Now we can let (i m ax> jmax) be a maximal square in L. Since p[i max , jmax] > 

7r[?max] jmax] ^C have ^max 

< n and j max > 1- There must be a black rook weakly 
to the right of (i max + 1, jmax) in row i max + 1 because otherwise the greater square 
(i mffl + 1, jmax) would belong to L. By an analogous argument, there is a black rook 
weakly above (« m ax, jmax — 1) in column j max — 1- We have the situation depicted 
in Figure [Tj 

Since there are more white than black rooks inside the rectangle R — [1, i max ] x 
[jmax> rt ] there must also be more white than black rooks inside the diagonally op- 
posite rectangle R' = [i max + l,n] X [1, jmax — 1]- In particular there is at least one 
white rook inside R and at least one white rook inside R' . Since p is covered by 
Hr(tv) it follows that there is a black rook (i, j) inside R and a black rook (i',f) 
inside R'; choose minimal in R'. Call (i, j) and the witnesses. Now the 
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Figure 8. The four cases of the "if" part of the proof of Theo- 
rem The shaded regions contain no black rooks. 



situation is exactly as in the proof of Theorem 4.2 in [H] by Gasharov and Reiner. 
The remaining part of the proof will essentially be a copy of their arguments. 

We show that at least one of the four forbidden patterns will appear, depending 
on whether the rectangle x contains a black rook strictly to the left of 

column jmax; and a black rook strictly below row i max . If one can find 

(1) both, then combining these with the two witnesses produces the pattern 
4231 in 7T. (Look at Figure |H] for illustrations.) 

(2) the former but not the latter, then combining the two witnesses with the 
former and with the black rooks in column j max and in row z max + 1 produces 
the pattern 42513. 

(3) the latter but not the former, then combining the two witnesses with the 
latter and with the black rooks in column j max — 1 and in row z max produces 
the pattern 35142. 

(4) neither, then combining the two witnesses with the black rooks in column 

Jmax 1 and Jmax 

and in row i max and z max + 1 produces the pattern 351624. 

□ 



5. POINCARE POLYNOMIALS OF A n 

In this section we apply Theorem 13.11 to the lower Bruhat interval [id, w] of the 
symmetric group A n where w is the maximal minimal coset representative w = 
max^f^ 5 *^. In the end we obtain the simple formula of Theorem 13.51 

Let J m ' n denote the m x n matrix with all entries equal to one. The following is 
a g-analogue of the corollary to Theorem 1 in |3] . 
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Proposition 5.1. Let A and B be zero-one matrices of sizes m x m and n x n, 
respectively. The block matrix 

R n 4 45. f ( B J n ' m 
ft I jm,n 

has the (m + n)th q-rook number 

min(m,?i) 

Proof. It is easy to see that each configuration tt of m + n rooks on Bj^A is chosen 
uniquely by the following procedure: 

• First, choose a nonnegative integer i. 

• Then choose a configuration A of m — i rooks on A and a configuration B 
of n — i rooks on B. Together A and B form a configuration of m + n — 2i 
rooks on ( ^ ^ ) • 

• Let X be the i x i submatrix consisting of the remaining free one-entries of 
( o ^o™ ) ' i ,e- t ne one-entries whose row and column have no rook in A or 
B. Similarly, let Y be the i x i submatrix consisting of the remaining free 
one-entries of ( jM, n g ) . Now choose a configuration X of i rooks on X and 
a configuration y i rooks on Y . 

Let Inv(7r) be the set of inversions of tt, i.e. pairs (r, r') of rooks such that r is strictly 
north-east of r' . The number mvA(A) counts the cells in A which have no rooks to 
the right or below. This equals the number of inversions (r, r') such that r belongs 
to A or J n ' m and r' belongs to A or J m ' n : 

inv^) = |{(r,r') 6 Inv(7r) : r e (g r ' e ( ^ o) } |. 

Similarly, inv s o (#^) counts the cells in B which have no rooks to the left or above, 
so 

mv B0 (B°) = |{(r,r') G Invfr) : r G (f ^ m ), r' € (ji, n °)}|. 
We also have 

inv x (^) = |{(r,r') Glnv(vr) : r,r' G ( /,n 

and 

invr(^) = |{(r,r / )Glnv( 7 r) : r,r'e(»7)}|. 
Putting the above equations together yields 

inv A (A) + inv B o (B°) + mv x (X) + invy (y) 
(2) = inv(^) + |{(r,r')Glnv(vr) : r G (g r' G ( j°, n g)}| 

= inv(7r) + i 2 

where inv(7r) = |Inv(-7r)|. Now we exponentiate and sum over all permutations tt 
which can be constructed by the procedure above: 

min(m,n) 
TT i=0 

By Corollary El Rf (?) = R- '(<?) = W-g- D 
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Proof of Theorem \S.5[ A Coxeter system of type (A n _\, S = {s±, S2, ■ ■ ■ , s n _i}) (see 
Figure 0} is isomorphic to the symmetric group & n with the adjacent transpositions 
Si = (i i + 1) as generators. A permutation w G 6 n can be represented by a rook 
configuration on J n ' n with n rooks, so that w(i) = j if and only if there is a rook in 
the cell 

Let w be the maximal element in A^i k \ i.e. (w(l), w(2), . . . , w(n)) = (n — k + 
1, n — k + 2, . . . , n, 1, 2, . . . , n — k) . Then we have Hr(w) = (Ti^_ fc #Tfc)I and hence 

Poin [idH (g) = i?i efc#Tfc)1 (g) = q^)£^* Tk {l/q) 
which by Proposition 15. II equals 

min(fc,n— fc) 

Using Equation Q we obtain 

min(fc,n— A;) 

P( g ) = ^ g-C l ^)s n _ fc+liJ+1 (l/ (7 )g-(") < S fc+1 , J+1 (l/ (/ )[ i ]!2 /g /. 

i=0 

Since = [i]\ q ■ q~^ we are done. □ 

6. The number of elements in a lower interval of A n 

For a general minimal coset representative w E Az^ k ^ , it seems very hard to 
compute the complete Poincare polynomial. In this section we will solve the easier 
problem to determine Poinr id M) i(l), i.e. the number of elements of [id, w\. We obtain 
a recurrence relation that allows us to count the elements in polynomial time. In 
the special case when w is the maximal element in AZ}j^ k ^ this method results in 
a formula different from what we get if we put q = 1 in Theorem 13.51 From this, 
rather unexpectedly, we obtain an identity of Stirling numbers that we have not 
seen in the literature. 

The set of minimal coset representatives A^^ k ' consists of the permutations 
w G & n with w(l) < w(2) < • ■ ■ < w(k) and w(k + 1) < w(k + 2) < • • • < w(n). 
Such a permutation clearly avoids the patterns in Corollary 13.31 so the number of 
elements in the Bruhat interval [id, w] is given by the nth rook number Rn R ^ ■ 
Fortunately Hji(w) has a simple structure. If w(k) < n then, as can be seen in 
Figure El 

(w(w(k) + l),w(w(k) + 2), . . . , w(n)) = (n — w(k) + 1, n — w(k) + 2, . . . , n), 

so the interval [id, w] is isomorphic (as a poset) to the interval [id, if'] in A w ^_i, 
where w'(i) = w(i) for i = 1, 2, . . . , w{k). Thus we may assume that w(k) = n. 
Then Hr(w) = A/u, where A and u are right-aligned Ferrers matrices with row 
lengths 

n if 1 < i < k, 

n — w(i) + 1 if k + 1 < i < n, 



n(A) 



w(i) if 1 < i < k, 

if k + 1 < i < n. 
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Figure 9. If w(k) < n as in the left example, we may instead study 
the smaller example to the right. They have isomorphic lower inter- 
vals. 



For 1 < i < k, let Pi be the n x n zero-one matrix with ones in the cells 
(i, 1), (i,2), . . . , (i,w(i)). It is easy to see that a rook configuration with n rooks 
is covered by \j \x if and only if it is covered by A and not by any A — P%. Thus, by 
the principle of inclusion-exclusion we get 

R ^= J2(-l) lIl Rn~ Ui5lPi - 
IC[k] 

By a suitable permutation of the rows, the matrix A — Uj e /Pj can be transformed 
to a Ferrers matrix v with column lengths 

Cj(y) = c,(A) - |{* £ / : w(i) > j}\ = Cj {\) - \w(I) n [j,n]\, 

where w(I) = {w(i) : i € /} denotes the image of / under w. Theorem 12 . 1 1 with 
x = gives 

n 

r^=r^(o)= (-i) |j| n^( A )-i jn ^>]i-j +i )- 

JCw({k]) j=i 

As we will see in a moment, this expression can be computed efficiently by dynamic 
programming. 

For 1 < a < n and < b < n, let 

n 

/m)= e n^( A )-i jn ^ n ]i-j +i ) 

j g ^([fc])n[a,n]^ j=a 

where ^M^) denotes the set of subsets of w([k\) D [a, n] of size b. Also put 
f(a, b) = if b < 0. It is straightforward to verify the following recurrence relation: 

(3) 

7M) = (aa(A)-a-6+l)/(o + l,6) ifn>o^«;([fc]), 
< /(a, 6) = (c a (A) - a - 6 + 1) (/(a + 1, 6) + /(a + 1, 6 - 1)) if n > a G «;([*;]), 

Here (5^^ is Dirac's <5-function which is 1 if b = and otherwise. Since 
(4) ^ = E("1) 6 /(1,6) 

6=0 
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the number of elements in [id, w] is computable in polynomial time. 

A special application of the method above admits us to prove our by-product 
Theorem O 

Proof of Theorem \3. 7| Consider the case when w is the maximal element in A^_ 1 Sfc ^, 
i.e. (w(l),w(2), . . . , w(n)) = (n — k + 1, n — k + 2, . . . , n, 1, 2, . . . , n — k) . Then 
c Q (A) = k + a if a<n — k and c a (A) = n if a > n — k + 1, so the recurrence © 
becomes 

f(a,b) = (k-b + l)f(a + l,b) if a <n-k, 

f(a, b) = (n-a-b+ l)(/(a + 1, b) + f(a + 1, b - 1)) ifn-fc + l<o<n-l. 

Iteration of the first line of the recurrence yields 

(5) /(l, b) = (k — b+ l) n - k f(n -k + l,b). 

Putting g(a, (3) = f(n — a + 1, a — 0) for 1 < a < k, our recurrence transforms to 

(g(a,f3) =(3- (g(a - 1,(3) + g(a - 1,(3 - 1)) if 2 < a < k. 
\g(l,(3)=6p > i 

We recognise this as the recurrence for (3\S ay p where S a ^ are Stirling numbers of 
the second kind; thus f(a, b) = (n — a — b + l)!5„_ a _|_i jn _ a _b + i for n — k+ 1 < a < n. 
Combining this with Equation ((SJ) and plugging the result into Equation we 
obtain 

k 
6=0 

which also can be written as 

k 

(-i) fc ^(-i) i (i+ir-^!5 fci4 . 

This happens to be the formula for the poly-Bernoulli number B^~ n defined by 
Kaneko [Eg. □ 

7. Type B 

In this section we compute the Poincare polynomial of the lower Bruhat interval 
[id, w] in the hyperoctahedral group B n , where w is the maximal minimal coset 
representative, w = maxBf^ s °^. 

We will represent B n combinatorially by the set 6^ = f {tt G &2n ■ tt° = ^} 
of rotationally symmetric maximal rook configurations on J 2ra ' 2n ) se e ^ Ch. 8]. In 
this representation tt < p in Bruhat order on B n if and only if 7r < p as elements of 
&2n (0 Cor. 8.1.9]). The rank of vr is 

(6) £{ir) = (inv(vr) + neg(7r))/2 

where inv(7r) is the usual inversion number of tt as an element of &2n, and neg(7r) = f 
\{i G [n + l,2n] : ir(i) < n}\, see Ch. 8, Exercise 2]. 
For a zero-one matrix A of size 2n x 2n, let 

RB A (q,t)= g inv(7r) t ncg(7r) . 
Treesns(A) 
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Proposition 7.1. Let A be a zero-one matrix of size n x n. Then 

n 

RB A °# A (q,t)=Y,Rt l (<I 2 ) 

Proof. The proof is almost identical to the proof of Proposition 15.11 

When m = n, it is easy to see that the permutation ir constructed by the proce- 
dure in the proof of Proposition 15.11 is rotationally symmetric if and only if B = AP 
and y = X°. Thus, putting m := n, B := A , Y := X° = X, B := A , and 
y := <Y° into Equation Q and using the identity (j4°)° = A, we obtain 

2 (iuv A (A) + mv x (X)) = inv(vr) + i 2 . 

Obviously, neg(-7r) = i. Exponentiation and summation over all rotationally sym- 
metric permutations it on A C ^#A yields 

n 

7T i=0 

By Corollary [T3] R? (q 2 ) = [»] ! g2 . □ 

Now we are ready for the proof of Theorem VAM 
Proof of Theorem I.V.61 From Theorem KS.ll and Equation Q we obtain 
Poin [id)W] (g) = £ / (W) =RB^)(?VV/ 2 ). 

«G[id,«i] 

It is easy to see that H R (w) = (Tp#T n )l. Hence 

where we have used the fact that 

RB Al (q,t) =q( 2 ?)t n RB A (q-\t- 1 ) 
for any 2n x 2n zero-one matrix A. By Proposition 17. II we can now compute 

n 

Poin [id)M] (g) = «f 2 £<l f (l/?)[t1! a/ , A 

i=0 

Using Equation Q and the identity [i]!i/ g g( 2 ) = we finally obtain 

n 

Poin [id)m ](g) = g( 2 ) ^2s n+hi+ i(l/q)[i][ q . 

□ 

8. Open problems 

Perhaps the reason we still do not know much about the Poincare polynomials of 
Bruhat intervals after several decades of research in the area is the lack of natural 
methods to attack it. We hope the framework and the tools presented here will make 
the problem more accessible, and we would like to suggest a number of interesting 
open questions. 

• What is the Poincare polynomial Pomp^] (q) in the even-signed permutation 

group D n if w = maxZ?f^ s °^ is the maximal minimal representative in the 
quotient modulo a maximal parabolic subgroup isomorphic to A n _i? 
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• What is Poin[ id lu ](g) in the affine group A n 7 

• Are there formulas for the generalized Eulerian polynomial ^„e[id w ] ^ or 
even for the bivariate generating function X^epdw] t d ^q^ v \ where d(v) = 
\{s € S : £(vs) < £{v)}\ is the descent number of vl 

• In a recent paper by Bjorner and Ekedahl 2 it is shown (for any crystallo- 
graphic Coxeter group) that < i < j < £(w) — i implies ff < where 
ff is the -coefficient of Poin[ id u ,](g). Perhaps one can say more about the 
particular Poincare polynomials discussed in the present paper. Are they 
unimodal, for instance? 

• As noted by Gasharov and Reiner [9, p. 559], Ding's partition varieties [Jj 
correspond not only to certain Bruhat intervals of the whole group A n but 
also to some intervals of the quotient A n /{A n )j for certain parabolic sub- 
groups (A n )j. Can something similar be done in the more general setting 
of skew partition varieties? 

• Given a polynomial, what board (i.e. zero-one matrix), if any, has it as 
its rook polynomial? In a recent paper Mitchell showed that distinct 
increasing Ferrers boards have distinct rook polynomials. What is true for 
skew Ferrers boards? 

• Develin [§] classified the isomorphism types of lower Bruhat intervals of 312- 
avoiding permutations by using the connection to rook posets discovered 
by Ding What are the isomorphism types of lower Bruhat intervals of 
permutations that avoid the patterns 4231, 35142, 42513, and 351624? 
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